This document explores the results of expert opinion gathered from staff at the Woodland Trust, related to the measurement of woodland ecological condition by a set of 14 indicators.

Methods

Expert opinion elicitation using the Delphi method.

Here, we estimate the value function and weight for each indicator using a method of expert opinion elicitation called the Delphi method. The Delphi method is designed to collect and distill expert knowledge, using iterative or repetitive surveys to increase robustness . The method is also useful for identifying where agreements and disagreements occur.

The method relies on repetitive surveying in at least two to three‘rounds’ where participants are asked the same questions, to which they provide quantitative answers. After each round participants review and consider the answers given by all panel-members, and are then given the opportunity to revise their answers. Results are considered relatively robust when they change little between rounds. The Delphi method has a history of successful use in supporting conservation action where empirical evidence is insufficient and information is needed rapidly to inform decisions. It’s transparency and repeatability also add to its apeal.

Survey questions

Each participant either attended an introductory workshop to the process, or was taken through individually. A sub-section of participants were also involved in development the proposed methods to measure each indicator in the field. Following this participants were supplied with an Excel spread sheet in which to submit their answers to questions relating to:

  • Value functions describing how each indicator typically related to WEC
  • Weightings decreeing the relative important of each indicator for WEC, typically
  • Qualitative comments relating to: situations where these “typical” responses might be inappropriate (exceptions), concerns with the proposed method to measure the indicator, and any other information though relevant.

Participants were encouraged to keep a clear distinction between:

  • The quantitative responses they provided regarding the relationship between indicators and WEC in a typical woodland
  • The exceptions to those typical relationships that they highlighted.

It was also stated that noted exceptions would be analysed and, where appropriate, used to inform future development of the WEC measure to account for important nuance.

Indicators considered

A set of 14 structural indicators were proposed to measure WEC.

Analysis

Scaling

Values and weights are all measured relative to each other, so each participant must be using the same scale. Some respondents did not use 100 and 0 as the highest and lowest scores for all indicator value functions and weights. To make their relative values comparable to other indicators all value functions provided where scaled to 0-100. Likewise, weightings were scaled so that the highest indicator weighting for each panelist was 100.

Value functions scaled for:

- Adam Th - Number deadwood classes present (of 3 potential classes) across 4 plot-quarters,
- Adam Th - Number of native tree and shrub species,
- Adam Th - Number of tree size classes (age proxy),
- Adam Th - Vertical structure: Number of tree and shrub canopy layers present,
- Alasdair Fi - Number deadwood classes present (of 3 potential classes) across 4 plot-quarters,
- Alasdair Fi - Number of tree size classes (age proxy),
- Alasdair Fi - Tree regeneration,
- Alasdair Fi - Vertical structure: Number of tree and shrub canopy layers present,
- Chris R - Number of ancient/veteran trees per 1 ha plot,
- Chris R - Number of tree size classes (age proxy),
- Dave Bo - Extent/ area of woodland,
- Dave Bo - Number deadwood classes present (of 3 potential classes) across 4 plot-quarters,
- Dave Bo - Number of ‘positive indicator’ plants per plot (10m radius circle),
- Dave Bo - Number of ancient/veteran trees per 1 ha plot,
- Dave Bo - Number of native tree and shrub species,
- Dave Bo - Number of tree size classes (age proxy),
- Dave Bo - Tree regeneration,
- Dave Bo - Vertical structure: Number of tree and shrub canopy layers present,
- Emma G - Number of native tree and shrub species,
- Emma G - Number of tree size classes (age proxy),
- Emma G - Occupancy of native trees & shrubs in all canopy layers,
- Jim Sm-Wr - Extent/ area of woodland,
- Jim Sm-Wr - Horizontal complexity (structural mosaics across a wood),
- Jim Sm-Wr - Invasive plant species presence and cover,
- Jim Sm-Wr - Number of ‘positive indicator’ plants per plot (10m radius circle),
- Jim Sm-Wr - Number of native tree and shrub species,
- Jim Sm-Wr - Number of tree size classes (age proxy),
- Jim Sm-Wr - Occupancy of native trees & shrubs in all canopy layers,
- Jim Sm-Wr - Vertical structure: Number of tree and shrub canopy layers present,
- Kylie Jo-Ma - Extent/ area of woodland,
- Kylie Jo-Ma - Herbivore impact,
- Kylie Jo-Ma - Number of ‘positive indicator’ plants per plot (10m radius circle),
- Kylie Jo-Ma - Number of ancient/veteran trees per 1 ha plot,
- Kylie Jo-Ma - Number of native tree and shrub species,
- Kylie Jo-Ma - Number of tree size classes (age proxy),
- Kylie Jo-Ma - Occupancy of native trees & shrubs in all canopy layers,
- Kylie Jo-Ma - Tree regeneration,
- Kylie Jo-Ma - Vertical structure: Number of tree and shrub canopy layers present,
- Liam Pl - Extent/ area of woodland,
- Liam Pl - Herbivore impact,
- Liam Pl - Number of native tree and shrub species,
- Liam Pl - Number of tree size classes (age proxy),
- Liam Pl - Occupancy of native trees & shrubs in all canopy layers,
- Liam Pl - Tree regeneration,
- Lou Ha - Number of tree size classes (age proxy),
- Martin Hu - Horizontal complexity (structural mosaics across a wood),
- Martin Hu - Number deadwood classes present (of 3 potential classes) across 4 plot-quarters,
- Martin Hu - Occupancy of native trees & shrubs in all canopy layers,
- Mick Br - Anthropogenic damage,
- Mick Br - Herbivore impact,
- Mick Br - Horizontal complexity (structural mosaics across a wood),
- Mick Br - Invasive plant species presence and cover,
- Mick Br - Number deadwood classes present (of 3 potential classes) across 4 plot-quarters,
- Mick Br - Number of ‘positive indicator’ plants per plot (10m radius circle),
- Mick Br - Number of ancient/veteran trees per 1 ha plot,
- Mick Br - Number of native tree and shrub species,
- Mick Br - Number of tree size classes (age proxy),
- Mick Br - Occupancy of native trees & shrubs in all canopy layers,
- Mick Br - Tree disease and rapid mortality,
- Mick Br - Tree regeneration,
- Peter Lo - Extent/ area of woodland,
- Peter Lo - Herbivore impact,
- Peter Lo - Horizontal complexity (structural mosaics across a wood),
- Peter Lo - Number of ‘positive indicator’ plants per plot (10m radius circle),
- Peter Lo - Number of ancient/veteran trees per 1 ha plot,
- Peter Lo - Number of native tree and shrub species,
- Peter Lo - Number of tree size classes (age proxy),
- Peter Lo - Vertical structure: Number of tree and shrub canopy layers present,
- Rich Br - Number of tree size classes (age proxy),
- Saul H - Extent/ area of woodland,
- Saul H - Horizontal complexity (structural mosaics across a wood),
- Saul H - Number of tree size classes (age proxy),
- Vanessa B - Herbivore impact,
- Vanessa B - Number of tree size classes (age proxy)

Weights were scaled for:


- Chris R,
- Dave Bo,
- Hannah Pa,
- Iain Mo,
- Jim Sm-Wr,
- Liam Pl,
- Saul H

Displaying of results

Between rounds, expert-opinion estimates were presented on interactive plots to allow respondents to investigate estimates provided by others, and their certainties.

Value functions where plotted with lines connecting each of the points provided, and a model (details below) was fitted to present the average trend, where a consensus was deemed to be emerging. Hover-text displayed respondent name, their certainty in their estimate, and a sentence provided that describes the trend.

Indicator relative weightings were presented on a dot-plot, including the mean and median results for each indicator’s weight.

Model details

Average value functions were modeled with a binomial GAM. The influence of points from each respondent weighted by the inverse of the number of points provided by that respondent, with the aim of ensuring that all respondents had the same influence on the result, regardless of how many points they provided.

Results - Round 1

Participants and survey completion

27 expert practitioners from Woodland Trust and Plantlife staff were invited to provide their opinion the value and weight of indicators.
20 have provided a response as of Tue Nov 21 16:27:28 2023.
Of these, 12 remain partially incomplete.
The chart below summarises participant completion of different elements of the survey. For more detail see table (button below chart).

Figure 1. Completion of expert opinion survey components by each panelist (coloured portion of pies) for value functions (“Value func”), value function certainties (“VF cert”), indicator weightings (“Weights”) and weighting certainties (“wt certs”).

Warning - table that should be printed here is written, just not printing

Value functions

Tree age/size distribution

[1] “Number of tree size classes (age proxy)” [1] “17 respondants.”

Categorised value function

measure Mean Standard deviation
1 0.0 0.0
2 40.0 15.5
3 76.5 15.0
4 100.0 0.0

Continuous Value function

Weights

Comments

Consensus on VF likely:
- All continuous relationships have same shape.
- Complete agreement on upper and lower value.
- Some put maximum value at 3 classes.

Disagreement on weight:
- Two camps, putting average weights at c. 60 and 90, respectively.

plot all indicators sequentially

Tree age distribution

Native canopy percentage

Vertical structure

N tree & shrub spp.

Invasive plants % cover

Deadwood

Veteran trees

Woodland extent

Regen

Herbivore damage

Tree health

Ground flora

Horizontal complexity

Anthropogenic damage

Indicatior weights

Certainties

Discussion points

  • - Some value functions have a minimum score >0. Strictly speaking this is okay… but it does meant that the final condition score will not be on a 0-100 scale (there will always be some minimum condition score).